Move snapshot data write to async mode. #299
Conversation
|
Codecov ReportAttention: Patch coverage is
❗ Your organization needs to install the Codecov GitHub app to enable full functionality. Additional details and impacted files@@ Coverage Diff @@
## main #299 +/- ##
==========================================
- Coverage 63.15% 61.14% -2.02%
==========================================
Files 32 34 +2
Lines 1900 3418 +1518
Branches 204 407 +203
==========================================
+ Hits 1200 2090 +890
- Misses 600 1109 +509
- Partials 100 219 +119 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
If a blob batch returns an error due to ALLOC_BLK_ERR, the async_write might still be in progress. |
xiaoxichen
force-pushed
the
br_opt
branch
2 times, most recently
from
f1f2fe8 to
0501da0
Compare
xiaoxichen
force-pushed
the
br_opt
branch
4 times, most recently
from
75a28df to
8b6d382
Compare
In order to provider higher queue depth for disk, which is optimal for scheduler to merge requests. Also IO and compute(checksum) can parallel. Signed-off-by: Xiaoxi Chen <[email protected]>
Signed-off-by: Xiaoxi Chen <[email protected]>
This make the function more closure and retryable. Signed-off-by: Xiaoxi Chen <[email protected]>
Signed-off-by: Xiaoxi Chen <[email protected]>
deleted and dup blobs should be counted. Signed-off-by: Xiaoxi Chen <[email protected]>
Signed-off-by: Xiaoxi Chen <[email protected]>
| bool success = | ||
| home_obj_.local_add_blob_info(ctx_->pg_id, BlobInfo{ctx_->shard_cursor, blob_id, blk_id}); | ||
| if (!success) { | ||
| LOGE("Failed to add blob info for blob_id={}", blob_id); |
There was a problem hiding this comment.
Do we need to free blk if commit_blk or add index fail?
| if (status != homestore::BlkAllocStatus::SUCCESS) { | ||
| LOGE("Failed to allocate blocks for shardID=0x{:x}, pg={}, shard=0x{:x} blob {}", ctx_->shard_cursor, | ||
| (ctx_->shard_cursor >> homeobject::shard_width), (ctx_->shard_cursor & homeobject::shard_mask), | ||
| blob->blob_id()); | ||
| std::unique_lock< std::shared_mutex > lock(ctx_->progress_lock); | ||
| ctx_->progress.error_count++; | ||
| return ALLOC_BLK_ERR; |
There was a problem hiding this comment.
Suggest to add a flip to simulate alloc error.
Signed-off-by: Xiaoxi Chen <[email protected]>
In order to provider higher queue depth for disk, which is optimal for scheduler to merge requests.
Also IO and compute(checksum) can parallel.